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ABSTBACT 

Multiple regression studies of schools are capable of 
detecting only "average" effect,, A basic assumption of such studies 
is that all schools are functioning within the parameters of a given 
educational technology and that differences in output can be 
attributed to different levels of input, . With decentralized decision 
making characteristics of public elementary and secondary education, 
it is entirely possible that some schools are functioning within 
totally different parameters than are :the bulk of schools. If these 
"different" schools are small in number, their ef feet _would be lost 
in a multiple regression analysis. Using data/of the public 
elementary schools in Hew. York State, regression analysis was used to 
identify outlier schools, socioeconomic characteristics being the _ 
independent variables and average pupil achievement being the 
dependent variable. Schools with an unusually large residual^ sc^ 
were considered outliers, and were compared with other schools using 
analysis of variance. Both positive and negative outliers tended to 
be small innovative schools, but the positive achievers were 
different with respect to resource applications in a direction which 
supports conventional wisdom about quality schools, . The paper 
concludes by discussing the implications for future research, (BH) 
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a?he last decade has produced a number of large scale statistical studies which 
have placed in question the effectiveness of traditional schooling arrangements. 



Some of the more frequently discussed arid' quoted include: Burkhead (1967); Coleman 
(1966); Hanushek (1970); Husen (1973); Jencks (1972); and Katzman (1971). All of 
these studies used multiple regression as the principal tool of analysis. a?hey 
pointed out that, on the average , the independent effect of schooling was small; 
yet within at least some of the samples - and probably all - there were schools 
which were unusually effective. Olhe purpose of this study was to identify, un- 
usually effective and ineffective schools frm the population of elementary schools 
in New York State and to examine them for distinguishing dif fereri&s in program, 
personnel and achievement characteristics. 

Disillusionment with multiple regression analysis as an appropriate tool for 
the study of scdiool effect has brought about a growing interest: in the potential: 
of studying outlying schools. Methodological considerations were discussed ex- 
tensively in a monograph by Robert Klitgaard and George Hall (1973). Noting that . 
large scale statistical studies have failed to identify any important and consistent 
school effects, they go on to suggest (1973:1): ^ 

"Perhaps educational research has looked in the wrong places, for evidence \ 
of effectiveness. Previous studies have indicated that, on average, school 
policies do not greatly affect measurable student scholastic and.occupa-. 



tionaj. performance. Suppose tiiis is true. Might there remain, nevertheless, 



a group of schools that are different? Are there any exceptions to 
small average tendencies and insignificant regression coefficients? 
The mathematics, of previous studies allow for such a possibility, as 
long as the number of exceptions is not large 

Klitgaard and Hall examined six data sets for outliers j Midhiigan schools ^ 
New York City elementaary schools. Project Talent, New York State school districts. 
New York State Schools (a liitiited sample of those which were included in this study) 
and Project Yardstick. They did identify groups of "over" achieving schools which 
comprised between two and nine percent of vsurious samples. Klitgaard and Hall 
(1973:71) concluded: 

"Moving away from average effects in educational research and policy making 
does seem worthwhile. We have located schools and districts that consis- 
tently perform better than their peers. It is probably worthwhile to 
continue such research, and to begin looking for unusucd.ly effective 
classrooms and programs." 

Implicit in the use of jmiltiple regression analysis to study school effect 
are assumptions of common objectives, comraon production functions, and competition 
among schools. The third assumption obviously does not pertain to piablic schools. 
With respect to the first two, there is ample evidence that there is much unifor- 
mity in practice as to the; purposes, organization and operation of schools. However 
with the extreme decentralization of decision-making in education and the absence 
of a science based technology, it would seem likely that at least a few schools 
would deviate substantially from the norms in terms of objectives and the applica- 
tion of resources. If highly successful or unsuccessful, such schools would emerge 
as statistical deviates or outliers. 
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procedures . The data for this study were collected by the Bureau of school 
programs Evaluation of the New York State Education Department (Irvine, 1976) and 
included information on 2,624 public and private elementary schools. This re- 
presents over 80% of the schools providing instruction to both third and sixth 

graders in the state. 

A conceptually simple criterion of school achievement (called the General 
Factor Score) was developed which was the sum of all average school achievement 
test scroes for reading and mathematics in grades three and nine for the 1972-73 
school year. An analysis by Nichols (1976) showed that such a procedure provided 
approximately equal weightings to the compoiient scores. 

A regression equation was calculated which predicted the General Factor Scores 
from seven variables describing the location and type of school and demographic 
characteristics of the school's student population. The means, standard deviations 
and correlation matrix of these variables are reported in Table 1 as are the re- 
gression coefficients and related statistics. The seven variables explained 69.6% 
of the variance in the achievement criterion. 

Using the raw score regression equation shown in Table 1, a predicted General 
Factor Score was calculated for each of the 2,624 schools. The predicted scores 
for each school was subtracted from the actual score to obtain a residual score. 
The residual scores indicated the degree to which schools* General Factor Scores 
were higher or lower than would be expected from student background variables. 

The distribution of residual scores had the following characteristics: 

Mean -00° 

Stcuideird Deviation 3.161 

Shew -.029 

Kurtosis -446 
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The slight negative skew indicates that the negative residuals tended to be 
more extreme than the positive ones, but this value is not statistically significant 
and is so small as to be unnoticeable in a plotted distribution. The positive 
kurtosis indicates that the tails of the distribution were longer than those of a 
normal distribution. A kurtosis of .45 would be slightly noticeable in a plotted 
distribution, and it is highly significant statistically with the large sample of 
schools (p .0001). Thus, the slight positive kurtosis was the only deviation of 
the distribution of residuals from a normal distribution with the expected mean 
and standard deviation. 

According to the central limits theorem one would eacpect a normal distribution 
of residuals if the schools' deviation from expected were due to the combined ef- 
fect of a large number of influences operating more or less independently of each 
other. The positive kurtosis suggests that there was also a fairly potent in- 
fluence operating on a relatively few schools to make their residual scores ex- 
treme. TSiis influence could be particularly effective or ineffective educational 
programs. However, it could also be some uncontrolled backgrovind factor or factors, 
or simply errors in the data. 

The residual difference between the actual and predicted school scores became 
the statistic for identifying school outliers. Schools which had a residual score 
in excess of 5 were classified as positive outliers. Those having a residual less 
than -5 were classified as negative outliers. Schools which had a residual score 
between .5 and -.5 were classified as non-deviates. 

The three groups were stibjected to a one way analysis of variance on 177 
variables relating to school pupil background, location, professional personnel, 
program, and achievement. Parallel analyses were made for private and public 
schools combined and for pviblic schools alone. 



Findings . One hundred forty-^ight schools were identified as positive out- 
liers ("over" achievers). Of these, 43 or 29% were private schools? lOS were 
public. In the negative outlier group ("under" achievers), there were 145 schools. 
Fifty, 34%, were private and 95 were public. In the non-deviate group, 104, 32%, 
were private and 219 were public. 

Table 2 reports the means and standard deviations for the three groups on the 
criterion variable and on the control variables (the independent variables in the 
regression equation used in predicting achievement) . As would be expected, there 
was little variation among the three groups within each ancOysis on the control 
variables. TSie total group averaged about 12% Negro enrollment with approxiinately 
1/6 of the schools exceeding 30%. Spanish JUnerican enrollment averaged over 6% 
with approximately 1/6 exceeding 20%. The pxiblic schools averaged more than 12% 
of their pupils on welfare; the percentage is lower for private schools. Private 
schools also tended to enroll a smaller percentage of students from minority groi^. 
on the achievement criterion, positive outliers averaged approximately one standard 
deviation above the total group mean; negative outliers averaged approximately one ' 
Standard deviation below. a?he average for non-^eviates fell near the total group 
mean. Private schools on the average achieved slightly higher than public schools. 

Table 3 lists those schools and program variables for which there was a dif- 
ference among group means which was significant at a .05 level or above. Both 
positive and^ negative outliers averaged about 100 students less than the non-deviates* 
Regardless of classification, private schools, averaging 220 pupils, were much smaller, 
than public schools, averaging 584 pupils. Generally, as the analysis shifts from 
the total group of schools to pxablic schools only, the direction of difference in 
school and program characteristics remains the same but in some ijistances the niagni-. 
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Table 3: Means and levels of significance of school and 
program variables which differentiate among the 
groups with a chance probability pf less than .05 



School and Program 


2 

Public and Private Schools 


Public 


Schools 0X1 




Variables 


Positive 


Non- 


N(?gative 


Positive 


Non- 


Negative 




Outlier 


Deviates 


Outlier 


1 Outlier 


Deviates 


Outlier! 


Total enrollment K-6 


426 


517 


402 


525 


637 


526 ■ 1 


Special Programs-^ 




+++ 


— 








* Academically talented"^ 


.257 


.149 


.083 


.210 


.187 




compensatory"^ 


.162 


.245 


.290 


.171 


.311 


.3053 


Building Arrangement 














Students/total room ratio 


16.80 


18.87 
+++ 


16.49 


18.60 


19.21 
+ 


17.87,: 


, 3 

Students/reg. classroom ratio 


0£ QQ 


+ 


^ i m / JU 


27.87 


28.89 


28.53 J 


% Total rooms reg. classrooms 


63.05 


65.52 


59.92 


66.89 


66.79 


62.5& i 


Organization-^ 




III 






+ 




Traditional classroom arrang. 


.939 


.957 


.890 


.943 


.973 


.905E 






+ 






+ 




Open classrooms^ 


.243 

" ++ 


.115 


.172 


.248 


.146 


.200? 


• 3 

Multi-age grouping 


.324 
+ 


.204 


.290 


• 267 


. 219 




Non-graded classes^ 


.223 


.164 


.317 
+++ 


.210 


.160 


.263i 


Gvddance counselors 


.270 


.341 


.461 


.276 


.411 


.58^ 








++ 









1 1 = program present^ 0 = program not present i Sl 

Individual group means that differ from the mean of all other groups combined are indicatedgl 
by + for high- means and - for low means . The nuniber of pluses or minuses indicate the P^Gb^ 
ability level: three = .001; two = .01; one = .05; and none = larger than .05. '■'^-^ '■^M 

!^ not significant for public schools only ^^^^^^ ^ ^^^^^ B 



tude of the difference drops. It would appear that private schools more than pub- 
lic schools tend to organize programs around the academically talented, multi-age 
grouping, and non-graded classes* 

Positive outliers were more likely to have programs for the academically 
talented, an open classroom arrangement and multi-age grouping. They were less 
likely to have compensatory education programs and guidance counselors. The num- 
ber of students per classroom was likely to be lower for both groups of deviates. 
Traditional classrooms were likely to be found in most schools. The chance for not 
finding them, however, was greatest in the negative outlier schools. The non-deviate 
schools were generally less likely to have instituted what might be termed "innovative 
programs than either of the deviate groups. 

The differences in achievement statistics was consistent for the four years 
studied, 1969/70 - 1972/73. Findings with reference to achievement are reported 
in Tcible 4 for 1971-72 only. For all years, all grades, and all subjects, achieve- 
ment means are high for the positive outlier schools and low for the negative out- 
lier schools. The standard deviations were smaller for positive outliers than for 
negative outliers for all years and for all subjects except sixth grade mathematics 
where the reverse was true for all years. As would be expected, the skew tests 
were positive for negative outliers and negative for positive outliers. Most dif- 
ferences on the achievement means were significant at the .001 level. 

Data on the dharacteristics of teaching staffs were available only for public 
schools. The findings are reported in^able 5. Positive outlier schools tended 
to have a smaller percentage of male teachers than did the other two groups (16, ; 
20, and 22% respectively). They also tended to have more highly trained staffs. : 
Non-deviates had the fewest uncertified teachers, 3.3^ while negative ov^i^Sr/y-^^^i-^i,;:/^^ 
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Table 4: Moans and levels of significance of achievement 

related variables for 1971-72 which differentiate 
among groups with a chemce probability of less 
than .05. 



Achievement Related 


Public and Private Schools"'' 




Public Schools Only^ 


Variables 


Positive 
Outliers 


Non- 
Devlates 


Negative 
Outliers 




Positive 
Outliers 


Non- 
Deviates 


Negative 
Outliers 


Means 
















Grade 3 - Readixig 


36.78 

+++ 


32.92 


27.28 




35.73 
+++ 


31.50 


25.79 


Grade 3 - Math 


36.96 
+++ 


32.25 


26.87 




36.24 
+++ 


30.97 


25.75 


Grade 6 - Reading 


46.00 

TTT 


41.29 


36.30 




45.06 
1 1 1 


39.30 


34.59 


Grade 6 - Math 


37.42 


32.35 


27.76 




37.00 

1 1 ■!' 


30.96 


26.53 


Standard Deviations 


+++ 














Grade 3 - Reading 


9.78 


10.36 


10.93 
+++ 




10.28 


10.84 


11.35 

+++ , 


«Grade 6 - Reading 


11.31 


11.82 


12.28 
++ 




11.90 


12.47 


12.77- 
+ 


Grade 6 - Math 


11.46 


10.69 


9.98 




11.83 
+++ 


10.98 


10.11 


Skew 
















Grade 3 - Reading 


-13.12 


- 4.66 


9.84 
+++ 




-12.58 


- 3.34 


13.13 ; 

+++ 


Grade 3 - Math 


- 7.25 


1.33 


9.50 

+++ 




- 7.11 


2.21 


12.03 

• 1 it, 1 


Grade 6 - Reading 


-14.85 


- 6.26 


1.36 
+++ 




-14.53 


- 3.62 


4.30 
+++ 


Grade 6 - Math 


.486 


7.07 


10.99 




1.27 


8.39 


12.68 








+++ 








"I" '1 '1' 



Individual group means that differ from the mean of all other groiips coiiODijied are ^^^^^^^/^^^ 
Indicated by + for high means and - for low leans. The number of pluses or m 
indicate the probability level: three = .001; two = .01; one = .05; axid none = 1^^ 
than .05. 
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had the most, 5.6%. Positive outliers averaged 4.1% uncertified teacSiers but they 
had the largest standard deviation, 10.7%. A larger percentage of teachers in 
positive outlier schools were on tenure than for the other two groups (69, 65 and 
59% respectively) . Teachers in positive outlier schools had more experience in 
the district and in total. The means in average total es^ierience for the three 
groups were 12, 11, and 10 years respectivfely. Teachers in positive outlier schools 
earned the highest salaries. Hhexr counter-parts in the negative outlier schools 
earned the least. 

Conclusions . The three groups of schools were traditionally oriented* however, 
outlier schools, both positive and negative, tended to be more innovative than non- 
deviate schools. For one group, traditional approaches and innovations were as- 
sociated with unpredictably high achievement; for another group, the opposite was 
true. What made the difference? Those who still believe that schools can make a 
difference in children's learning can gather some comfort from the analysis of j 
teacher characteristics. The teachers as a group in the positive outlier sdhools 
epitomized what conventional wisdom claims to be characteristic of good teaching 
staffs. They were better trained and more experienced. Oljey were paid Mgher-r^ 
salaries and were more likely to be on tenure. 

This information is of little practical value, however. It still does not 
inform as to what well trained, eacperienced and highly paid teacihers do to bring 
about unexpectedly high pupil achievement. Such knowledge is essential if we are 
to prepare and organize teachers and the teach '.iig processes; in other than a a^dom 
fashion. Suefli information cannot be gained by analyzing existing large scale data . ;|| 



banks. Understanding of the teachingAearxdng interface only fraoi 'y^r:^ 



an intimate study of interactions of persons involved in the process. 
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The studies of the past decade have generally followed an iX5>ut-output ap- 
proach, making few, if any, assxamptions about the interactions of variables. 
The conceptual wealaxess of this approach was recognized by Donald Levine in his 
analysis of Christopher Jendc's (1972) study of Inequality . Levine (1973:163) 
writes: 

"If effective educational practice is a matter of interactions , then 
attention must center on those ultimately responsible for structiiring 
and controlling the interactions; that is, teachers and students. The 
use that these agents make of the school resources usually measured in 
input-output analysis may be more significant than the absolute levels 
and kinds of resources." 
Jesse Burkhead (1973:204) the first economist to apply micro-economic techniques 
to the study of educational production fvuictionsr was probably the first to re- . 
cognize the substantial limitations of the application. In reviewing the results 
of economic reserach in education during the past 12 years r he writes: 

"Some, but certainly not all, economists who have worked on the micro- 
economics of education now feel that continued research in the cognitive 
domain is largely fruitless unless it is somehow combined witix research 
in the affective domain outcomes. This is the responsibility of psycholo- 
gists r sociologists, and educators, not of econcMnists . . . The difficulties 
. of this kind of research are siabstantial. The first is a lack o£ communica-. • 
tion among educators, psychologists, cuid sociologists.** 

The literature in anthropology , psychology, and sociology as well as in educa- 
tion contains many references to interaction and social climate type -factors 
may be linked to pupil achievement. Gearing et al . (1973), Gearing and Tindall (1973) 
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Brophy and Good (1970), Rist (1970), Rosenfeld (1971) and TaUbert (1973) have 
analyzed arrays of formal and informal social identities and corresponding arrays 
of established role elaborations among these. Cole and Miller (1967) foxind that 
achievement oriented .values predicted grades independently of ability and Gross 
(1967) observed that value conflicts between teacher and pupil may have a deterent 
effect on achievement. Self-esteem is related to effective performance in a given 
setting according to Fitts (1965) as is the pupil's perception of control over his 
behavior and its consequences, his sense of competence, and the amount of anxiety 
experienced in classroom activities. 

Silberman (1970) has noted that classrooms differ structually on several dimen- 
sions such as the amount of choice pupils have in determining classroom activities, 
the frequency of group activities which require the cooperative use of various com- 
munications skills to solve common problems, the niomber of activil^ centers in the 
classroom, and whether the teacher works mainly with the class as a vAiole or with 
individuals and small groups. Schmuck and Schmuck (1971) identified six dimensions 
of social climate: leadership, patterns of attractions and liking, classroom norms 
related to work and social behavior, freedom of openness of communication, group 
cohesiveness, and the legitimacy of open eveiluation of classroom activity. Scheff 
(1963) noted a linkage between formal labeling and deviant behavior and the nuxnber 
of alternatives available to handle unusual behavior. Bartlett (1972) studied re- 
lationships between teaching sl^le and deviemt heavier. 

The general literature on organizations Le.g., Larsch and Lawrence (1970)^ 
Mott (1972), Paylor and Bowers (1972), and Tanneribaum (1968)J has implications for 
school organizations. Teacher perceptions of t^eir relationships within heirar<dii- 
cal and peer networks, communication patteims, the distribution of influence by 
decision* areas, orientations to tasks and to role, and the ciiaracter of task support 

■ . , 15 



systems may well affect the way teachers relate to their schools and to mission 
accomplishment • 

The studies identifying these factors have been highly fragmented and no 
attempt has been made to weave them into a single conceptualization. After a very 
careful review of the extensive research on educational effectiveness, the Rand 
Corporation in a report to the President's Commission on School Finance (Averich 
et al., (1971:165)) criticized that research by noting that \^ile scholars from 
several disciplines study educational phenomena, they follow "relatively narrow, 
intra-disciplinary paths. There have been few attempts to connect these paths; 
nor is there a clear map down any given pc\th." To develop the complex experimental 
designs necessary to unravel the puzzles of educational interaction effects, they 
observe that "it will be necessary to merge the various research appro^^fK^^ . 

in summary, it would appear that the research of the past decade related to 
schooling, learning and teaching has several inherent shortcomings. 

1. On the basis of questionable assumptions, schooling research has in- 
variably looked for an average effect rather than isolate and analyze 
those schools which are unusually effective or ineffective. - 

2. The research has used measvires of resource ii^ut as independent vari- 
ables rather than measures describing their interaction (the teaching/ 
learning interface) . 

3. The conceptalizations guiding the interaction research have been uni- 
disciplinary and inadequate to account for the very large number of 
forces operatixig in a classroom. 

Before significant advancements can be made in the understanding of the 
teaching/learning interface, it will be necessary tp integrate the conceptualiza- 
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tions and to adapt the research approaches of anthropology, psychology, sociology 
and education into ones which are Capable of describing and analyzing the complexities 
of the educational process. Despite their noteable lack of success in the past, such 
a task could not be accomplished by other than multi-disciplinaacy reseeucch teams. 
The in depth study of the teaching/learning interface in outlier schools would pro- 
vide an unusually interesting and stimulating problem on which multi-disciplinaacy 
teams cduld focus. 
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